Skip to content

Declare tables as objects, so constraints have somewhere to live - #137

Merged
jbylund merged 1 commit into
mainfrom
feat/declared-table-and-schema
Aug 17, 2026
Merged

jbylund merged 1 commit into
mainfrom
feat/declared-table-and-schema

Conversation

@jbylund

@jbylund jbylund commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Closes #126.

Session.schema() returned {table_name: {column_name: type_str}}, which describes columns and nothing else — so there was nowhere to say that commits.sha is a primary key or that commit_files.sha references it, and therefore nowhere for pg_constraint and pg_index to get rows from. This adds the objects that have somewhere to put them and deliberately nothing else: no constraint can be declared yet.

class MySession(Session):
    async def schema(self):
        return Schema([Table("users", {"id": "integer", "name": "text"})])

The nested dict is still accepted, forever — resolve() normalises it.

Two invariants now written down and tested

Both were implicit and both would have died quietly to a later tidy-up:

  • Table order decides table OIDs (_FIRST_USER_OID + index) and column order is ordinal_position and attnum. Sorting either mapping renumbers something a client may already have introspected.
  • A Table is frozen with its columns copied into a read-only mapping. session_factory runs once per client, so a module-level Table is shared by every connection — and frozen=True alone would still allow table.columns["sha"] = "integer".

Shapes refused on purpose

Schema holds dict[str, Table] built from Table.name rather than accepting a mapping, so a name and its key cannot disagree and a duplicate raises. resolve() refuses Mapping[str, Table] for the same reason — it is the one shape where a table could end up with two names. Everything reaches the tables through the single read-only .tables accessor, so Schema grows no partial collection protocol that could be mistaken for the old dict.

The one deliberate behaviour change

TableSession.schema() returns a Schema. Not a simplification — the real one is retiring tables.py's private _Table, which needs #134 first (#135) — but pg_mimic's own session should not hand back the legacy shape once Schema is canonical. Schema.column_types() is the same declaration in the old shape.

Session.schema() also defaults to an empty Schema rather than None, which deletes the branch whose mis-binding once crashed every catalog query for any session not overriding it. resolve() still tolerates None, because an override may return one and test_catalog.py has a session that does.

This collapses "I declare no tables" and "I make no claim about my tables" into one value; #136 covers whether that is worth distinguishing again.

Also here, because the refactor touched every call site

catalog.py used "schema" for three different things, and _build_information_schema took one sense and returned another — schema, tables = _build_...() read as though the first contained the second, when it was really sqlglot's schema= and tables=. That pair is now a named _CatalogInput(column_types, rows).

git_sql is converted as the first real consumer

An API with no caller is untested design. The conversion surfaced exactly the entanglement #135 will fix: git_sql's one dict served as both the declaration and sqlglot's schema= argument, so it now derives COLUMN_TYPES = SCHEMA.column_types() once. Verified live — \dt, \d commits and a real query all answer as before.

Tests

tests/test_declared.py is 22 new tests, aimed at the properties nothing else would notice breaking: order, immutability, aliasing, and which shapes resolve() refuses. Three existing test files construct catalog input directly and now pass resolve(...); test_schema_is_derived_from_the_tables asserts through column_types() and additionally pins the table order.

Full suite: 908 passed, 32 xfailed. ruff check and ruff format --check clean.

Not in scope

Primary keys (#127), unique constraints (#128), foreign keys (#129), late declaration (#130), rows on a Table (#131, #132), the OID scheme (unchanged — positional and deterministic), and type-spelling validation (#134, with a TODO in Table.__post_init__ marking where it goes).

Session.schema() returned {table_name: {column_name: type_str}}, which describes
columns and nothing else -- so there was nowhere to say that commits.sha is a
primary key or that commit_files.sha references it, and therefore nowhere for
pg_constraint and pg_index to get rows from. This adds the objects that have
somewhere to put them, and deliberately nothing else: no constraint can be
declared yet, and no behaviour changes but the one noted below.

pg_mimic/declared.py is a leaf -- catalog, copy and tables all import it and it
imports nothing back, because #92 records the one real import cycle this package
has and this is not joining it. resolve() is the only thing that reads
schema()'s return value, since it is the single place that knows every shape a
session may hand back: a Schema, a sequence of Table, the historical nested
dict, or None.

Two invariants are now written down and tested rather than implicit. Table order
decides table OIDs (_FIRST_USER_OID + index) and column order is
ordinal_position and attnum, so sorting either mapping renumbers something a
client may already have introspected. And a Table is frozen with its columns
copied into a read-only mapping, because session_factory runs once per client:
a module-level Table is shared by every connection, and frozen=True alone would
still allow table.columns["sha"] = "integer".

Schema holds dict[str, Table] built from Table.name rather than accepting a
mapping, so a name and its key cannot disagree and a duplicate raises. resolve()
refuses Mapping[str, Table] for the same reason -- it is the one shape where a
table could end up with two names -- and everything reaches the tables through
the single read-only .tables accessor, so Schema grows no partial collection
protocol that could be mistaken for the old dict.

The one deliberate behaviour change: TableSession.schema() returns a Schema.
Not a simplification -- the real one is retiring tables.py's private _Table,
which needs #134 first -- but pg_mimic's own session should not hand back the
legacy shape once Schema is the canonical one. Schema.column_types() is the same
declaration in the old shape. Session.schema() also defaults to an empty Schema
rather than None, which deletes the branch whose mis-binding once crashed every
catalog query for a session that did not override it; resolve() still tolerates
None because an override may return one, and one in the suite does.

Also here because the refactor already touched every call site: catalog.py used
"schema" for three different things, and _build_information_schema took one
sense and returned another, so `schema, tables = _build_...()` read as though
the first contained the second when it was really sqlglot's schema= and tables=.
That pair is now a named _CatalogInput(column_types, rows).

examples/git_sql.py is converted as the first real consumer -- an API with no
caller is untested design -- which surfaced the entanglement #135 will fix: its
one dict served as both the declaration and sqlglot's schema= argument.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jbylund
jbylund merged commit fea1da1 into main Aug 17, 2026
6 checks passed
jbylund added a commit that referenced this pull request Aug 17, 2026
Closes #127. **Stacked on #137** (`feat/declared-table-and-schema`) —
review that first; this PR targets its branch, so the diff here is only
the primary-key work.

`Table` takes `primary_key`, one column or a tuple of them:

```python
Table("commits", {"sha": "text", "author": "text"}, primary_key="sha")
Table("commit_files", {"sha": "text", "path": "text"}, primary_key=("sha", "path"))
```

Normalised to a tuple so nothing downstream branches on the spelling,
and validated at construction against the table's own columns — a key
naming a column the table does not have is knowable there, so it raises
on the line that wrote it. A tuple rather than a set because a composite
key's order is part of it: it is what `conkey` records and what psql
prints.

## What `\d` prints now

```
              Table "public.commit_files"
   Column   |  Type   | Collation | Nullable | Default
------------+---------+-----------+----------+---------
 sha        | text    |           | not null |
 path       | text    |           | not null |
 insertions | integer |           |          |
 deletions  | integer |           |          |
Indexes:
    "commit_files_pkey" PRIMARY KEY, btree (sha, path)
```

Byte-for-byte what PostgreSQL 18 prints for the same declaration — I
diffed them against a real server rather than reconstructing the format.

## Four things that are each invisible on their own

This is most of the work, and none of it is guessable from the outside:

- **`pg_class.relhasindex`** gates the entire footer. Left `False`,
every row below is built and never asked for.
- **The index needs its own `pg_class` row**, `relkind = 'i'`. psql
reads the index's name as `c2.relname` off a *second* `pg_class` in the
same query, so without it the join drops every index it was told about.
- **`pg_get_indexdef` cannot be the constant `""`** the other catalog
functions are. psql echoes everything after `" USING "` straight into
the footer, so that substring is where `btree (sha)` comes from. It is
rewritten into a reference to a column carrying the rendered text —
exactly as `format_type` already was — and `pg_get_constraintdef` gets
the same treatment, which #129 will need.
- **`pg_index` and `pg_constraint` leave `_ALWAYS_EMPTY_TABLES`.** Safe
because psql reads both in a JOIN, not in one of the correlated
subqueries that substitution exists for; I checked every query `\d`
sends.

## Details worth a look in review

**`not null` is written in two places on purpose.** `attnotnull` and
`information_schema.columns.is_nullable` answer one question and a
client reads whichever it reads, so they must not disagree. There's a
test asserting both.

**Index and constraint OIDs come from their own ranges**, not the
continuation of the table numbering — declaring a key must not renumber
a *table*, since table OIDs are positional and a client may have
introspected them already.
`test_declaring_a_primary_key_does_not_renumber_the_tables` pins that.

**PostgreSQL 17 also records key columns as `contype='n'` rows in
`pg_constraint`.** pg_mimic reports `server_version 16`, so it
deliberately does not.

**Identifiers are quoted only when they have to be**, which is what
`pg_get_constraintdef` does — `btree (sha)` and `btree ("userId")`.
Reserved words are not consulted; the comment says why.

## Tests

20 new, at three levels:

- `test_declared.py` — normalisation, key order, and the two refusals.
- `test_pg_catalog.py` — **psql's index-footer query verbatim**, against
one-column, composite and needs-quoting keys, plus a table with no key
answering no rows. Also the OID-stability and `conkey` assertions.
- `test_psql.py` — the real psql binary, asserting the rendered footer
string, that a keyless table has no `Indexes:` section, that `\di` lists
the index, and that `\dt` does *not* — an index is a relation, so the
`pg_class` row it needs must not leak into the table list.

Full suite: **928 passed, 32 xfailed** (up from 908). `ruff check` and
`ruff format --check` clean.

## Also here

`examples/git_sql.py` declares the four keys that are actually sound for
git data: `commits.sha`, `files.path`, `branches.name` and `commit_files
(sha, path)`. Verified live.

## Not in scope

Unique constraints (#128), foreign keys (#129), declaring a constraint
after construction (#130). `pg_get_constraintdef` is made row-dependent
here but no non-primary constraint exists to render yet.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
jbylund added a commit that referenced this pull request Aug 17, 2026
Closes #128. **Stacked on #138**, which is stacked on #137 — this PR
targets #138's branch, so the diff here is only the unique-constraint
work.

`Table` takes `unique`, any number of keys spelled the way `primary_key`
is:

```python
Table("users", {"id": "integer", "email": "text", "tenant": "text"},
      primary_key="id", unique=["email", ("tenant", "email")])
```

Output is byte-identical to PostgreSQL 18 for the same declaration,
ordering included — psql sorts the footer `indisprimary DESC, relname`:

```
Indexes:
    "u1_pkey" PRIMARY KEY, btree (id)
    "u1_a_b_key" UNIQUE CONSTRAINT, btree (a, b)
    "u1_email_key" UNIQUE CONSTRAINT, btree (email)
    "u1_oddName_key" UNIQUE CONSTRAINT, btree ("oddName")
```

## Most of the diff is undoing a one-key-per-table assumption

#128 was supposed to be small, and the declaration half was. The catalog
half was not, because #127 had baked in "at most one key per table" in
two places:

- **Index and constraint OIDs came from the table's position**
(`_FIRST_INDEX_OID + index`), so a table with three unique constraints
would have issued three rows sharing one OID.
- **The row builder was a single `if table.primary_key:` block.**

Both are now driven by a running count over the rows actually emitted.
Worth knowing for #129, which adds a third kind of constraint to the
same loop.

## Where the two kinds of key actually differ

They share one code path that branches in exactly three places, which is
the useful summary:

| | primary key | unique |
| --- | --- | --- |
| `contype` / rendering | `p` / `PRIMARY KEY (...)` | `u` / `UNIQUE
(...)` |
| `indisprimary` | true | false — psql reads this to pick the footer
label |
| columns implicitly NOT NULL | yes | **no** |

The third is the one substantive difference, and it has a test asserting
`\d u1` says `not null` exactly once.

## Naming

Index names are *relation* names, so they are unique across the schema
rather than per table, and generated the way Postgres generates them:
`t_pkey`, or `t_<columns>_key` with the column names unquoted whatever
they are — Postgres builds `u1_oddName_key` from `"oddName"` and quotes
the whole name only when printing it.

Truncated to 63 bytes (NAMEDATALEN - 1), which is reachable with an
ordinary long table name plus two columns, and a truncation that
collides gets a counter as Postgres' own `ChooseIndexName` does.
Identical *keys* are refused at declaration instead, since that is a
copy-paste rather than a name clash.

## Tests

18 new. `test_declared.py` for normalisation and the three refusals;
`test_pg_catalog.py` running psql's footer query verbatim and asserting
the full ordered row set, plus the truncation and OID-uniqueness cases;
`test_psql.py` asserting all four rendered footer lines against the real
binary.

Full suite: **946 passed, 32 xfailed** (up from 928). `ruff check` and
`ruff format --check` clean.

## Not in scope

Foreign keys (#129) and declaring a constraint after construction
(#130). Unique *indexes* that are not constraints — `CREATE UNIQUE
INDEX` without `ADD CONSTRAINT`, which psql prints as `UNIQUE,` rather
than `UNIQUE CONSTRAINT,` — have no declaration surface and nothing
needs them.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A Table and a Schema for Session.schema(), so there is somewhere to declare constraints

1 participant